Electronic Publishing: Research Issues for Academic Librarians and Users

Increased reliance on electronic resources requires examination of the roles of librarians in several key ways. This paper addresses the need for further research into three important areas of electronic publishing. How is the change to digital information sources affecting the scholarly work of college and university students? Previous research shows that students rely on Web and online resources and ask for less help from librarians. We do not know, however, how these changes will affect the learning and scholarly work of college and university students. Research is also needed to determine how the differences between separate article and full journal databases affect the way research is done. What are the implications for scholarship of decisions being made about what publishers publish and what librarians purchase? Finally, are librarians—as intermediaries to the search process—still necessary in a digital age? Online systems are designed to be used independently but that may not always yield the best results. Introduction In the rush to a digital information world we rarely pause to consider the long-term effects on libraries, scholars, and students. Even more rarely considered are the long-term effects that changes in the media of scholarly communication may have on learning and understanding of content. Marshall McLuhan (1964) understood the relationship between variations in media and meaning over time and the impacts of these variations on society when he famously declared that the “medium is the message.” McLuhan was concerned mostly with television, radio, and mass media, rather than with scholarly information sources. It is difficult to know if Carol Tenopir, Professor, School of Information Sciences, University of Tennessee, Knoxville, TN 37996 615 tenopir/electronic publishing such concerns are valid for the materials that academic libraries emphasize—scholarly research reports, journals, and other reference and information resources that are used for finding information rather than for entertainment or news. If the same scholarly information is available in both printed and digital form, are there differences in how it is used and understood? Or, does the content stand separate from the medium through which it is delivered? How can librarians and other information professionals apply their knowledge about how scholarly materials are organized and used to help people make the most of the information available to them? Many research questions can be posed concerning this general issue, including: • How does the medium of information affect people’s preferences, understanding, and use of the content? • Do people seek for, understand, and use information delivered in digital form differently from that delivered on paper? • Does the medium of information resources used make a difference in the new knowledge people create from these information resources? • How does a library’s choice of format for information products affect the library users? These very broad questions need to be answered in the coming decade as libraries continue to move to digital services, but to answer such general questions, many more specific research questions must be addressed. This article explores three of the specific research questions that are critical to the delivery of effective library service to students and scholars. 1. Medium and Student Work: How is the change to digital information sources affecting the scholarly work of college and university students? 2. New Models for Scholarly Journals: When libraries select electronic journals, how do the products offered to them or the models they choose (such as a database of separate papers rather than a journal issue model) influence scholarship and the way journals are used? 3. Librarians as Intermediaries: Are librarians—as intermediaries to the search process—still necessary in a digital age? What do human information professionals bring to the online research process that cannot be adequately duplicated online? Question One: Medium and Student Work— How Is The Change to Digital Information Sources Affecting the Scholarly Work of College and University Students? This includes several subquestions, including: Are there differences in citing patterns, quality of work, and the research process? When faculty assign papers that require students to use the recorded knowledge, what 616 library trends/spring 2003 do they expect and how has this changed? How can educators and librarians influence students’ work in a positive way? Why This Question Is Important It is unclear how the move from paper-based library collections to digital collections is affecting scholarly work. In the past when faculty assigned research papers to their students, the expectations of what types of sources would be used and the role of librarians in providing access to these sources were fairly clear. Today, there is much more variety in media and sources available to students and students often do their research without setting foot in a library. Libraries purchase electronic resources and students access the resources from their dormitories, homes, or offices. Students may not seek help from librarians, either because they have confidence in their online searching skills or because they no longer come to the libraries and librarians are “out of sight, out of mind.” Librarians have always been concerned that college students demonstrate basic information literacy skills, but in today’s virtual library environments teaching these skills may be more important (and less controllable) than ever. In 2000 the Association of College and Research Libraries (ACRL) adopted competency standards for information literacy that recommend: students must be able to articulate information needs, know how to search for and access information resources, and effectively use the information they find (http://www.ala.org/acrl/standardlo.html). Librarians face new challenges in reaching students who only access library resources online and have grown up with the Web. It is difficult to make sure that such students know how to select appropriate resources, evaluate the quality of what they select, and use these resources well regardless of format or medium. Academic librarians and faculty members would likely agree that most college students today prefer digital resources to print resources. Even within the realm of digital resources, a majority of undergraduate students seem to choose Web sites or digital resources that include the full text often without regard to how appropriate the source is for their assignment. The impact of this is not yet known, but several aspects of this issue need to be studied. Librarians need to know more about how students make decisions regarding what information sources they use for their class assignments, in order to design better library instruction programs and so librarians can work more effectively with faculty and with students. If these decisions are based more on the convenience of electronic full texts or affinity for the Web, we need to understand how the change to digital information sources is affecting the scholarly work of college and university students. The prevalence of electronic resources is likely causing changes in students’ citing patterns and may be even changing the quality of work and the research process. Ultimately, understanding these changes in the work habits of 617 tenopir/electronic publishing undergraduate students may help educators and librarians influence students’ work in a positive way. Previous Work Student Preferences. In the late 1990s, Tenopir (1999) and Tenopir & Read (2000) recorded academic librarians’ observations of online use among students in all types and sizes of colleges and universities. These surveys found that appropriate content, recommendations from trusted people, and ease-of-use are all believed to be important factors that influence which research online resources are selected by students. The one factor, however, that was found to be most important was the availability of the full text. (In fact, availability of full text was mentioned by almost all respondents from every size of library.) According to one librarian from a small college: “Students expect to see the full article pop out. Also, full-text greatly enhances our own, small collection.” Another respondent explained, “The concept of one-stop information shopping is an important factor in using any electronic information resource” (Tenopir, 1999). Full text often overrides all other preference factors, in particular for undergraduate students. Librarians who have faith in users to pick the best content sometimes change their opinion when full text is the issue, and, increasingly, not just any full-text will do. One librarian summarized the new expectations: “End users want full text; preferably with graphics and they want it delivered over a Web interface that provides the flexibility for a variety of output and access options” (Tenopir, 1999, p. 38). When more than one full-text database is available, students may choose the resource that is best suited to their research needs, although convenience and recommendations by peers, instructors, and librarians are influential. Librarians reported in the Tenopir survey that when products are recommended specifically by their instructors or by librarians, students generally use them. Faculty Concerns. Many university faculty are concerned that students choose the convenience of full-text digital resources (and in particular, free Web sites) without regard to quality or appropriateness for their tasks. One recent study (Herring, 2000) found that although faculty respondents feel positive about the Web, they expressed concerns about their students’ ability to evaluate Internet resources and questioned the accuracy and reliability of Web resources. Rothenberg attributed a decline in the quality of student term papers—both in quality of writing and originality of thought—to reliance on the Web for research. In a recent survey of academic research libraries (Tenopir & Ennis, 2001, 2002) one respondent commented that “more and more faculty are unhappy with the Internet resources their students are using and have come to the librarians to help instruct students on reliable resources” (p. 270). Some students may not be aware of the need to carefully evaluate Web 618 library trends/spring 2003 and other full text sources. Grimes & Boening (2001) found that students are using Web resources in English composition assignments without evaluating the quality and reliability of the sources, contrary to the expectations of their professors. Preference for resources that are convenient and familiar is natural, of course. Not all faculty, and certainly not all students agree that the Web has had a negative effect on student work. McBride & Dickstein (1998) saw no decline in quality of student writing in Women’s Studies at the University of Arizona, even though the Web was used more often for research. A 1998 survey by librarians at Duke University found that 20 percent of undergraduate students interviewed felt the Web had a positive effect on their work, while another 40 percent felt the Web made a difference to the quality of their work. Over half identified the Web as a time saver (Lubans, 1998). D’Esposito & Gardner (1999) found that undergraduate students are aware that quality is an important factor to consider when selecting Web resources to use for schoolwork. Students in their study reported that government, educational, and reputable business sites are the most reliable. Undergraduate students studied by Wei He & Jacobson (1996) felt strongly that the Internet was useful for research and seeking information. Motivation. Motivations of students who use the Web were studied by Ko (2000) from the perspective of media uses and gratification. Ko (2000) found that students with higher motivation tended to go more to informational sites (in contrast to recreational or social sites), but she did not examine the quality or accuracy of the informational sites they visited. Duggan et al. (1999) asked students to self-judge “valuable educational” Web sites. They found that students who keep track of these sites have a more favorable attitude about using the Web for class assignments. Many of these studies imply that, contrary to faculty belief as measured in other studies, some students are aware of the need to judge quality of Web sites. The problem may simply be that they do not know how to do so. Instruction by librarians on how to evaluate topical Web sites and integrate them into classes may help solve this problem. It is not surprising that peoples’ motivation levels influence how much effort they put into a task and how long they stick with it. It has been well demonstrated that motivation plays a role in what medium users select (Lin, 1999). The use of digital research materials is no exception, whether the searchers are students, scholars, or scientists. Research by Nahl & Tenopir (1996) and Tenopir, Nahl-Jakobovits, and Howard (1991) found that time spent online and search strategies varied if faculty or university students were searching for personal or for workrelated purposes. More highly motivated users spent more time on a search and made more changes to their search strategies. Wildemuth et al. (1992) found that the search moves used by medical students varied with motivation (or what they called “search stimulus”). 619 tenopir/electronic publishing Motivation also plays a factor in what resources are selected. Ko (2000) discovered that when users are motivated to use the Web by the need to learn new things or to solve problems (considered by Ko as “information” motivations) they are more likely to visit “personal identity” and informational Web sites. Users who are online a lot are more likely to visit informational Web sites. Librarians and Recommendations. Librarians are making adjustments to student preferences for the Web. One librarian responded to the Tenopir and Ennis survey: “we have found in the last couple of years that too many students, because they know how to use the Web are using only the Web itself to find materials for their assignments, rather than trying the periodical indexes that we offer via the Web. We’ve had to change our instruction efforts to address this problem. We don’t discourage them from searching the Web for material on their subject in addition to using the indexes, but we talk a lot about the quality of what they find on the Web—the quality and the provenance” (Tenopir & Ennis, 2002, pp. 270–71). A librarian’s recommendation is believed by most academic librarians to be very important, although in most libraries trying to reach a large proportion of students and faculty is a constant effort. A database demonstrated in instruction classes or English 101 will be selected if students remember it when they need information. For those students who are motivated enough to ask at the reference desk, the recommendation of a specific source is believed to be very influential. Tenopir (1999) found that one library saw its usage statistics triple for a particular online system when they introduced a separate instruction class and promoted its use with faculty. Recommendations of relevant sources may be most important the first time a student has to choose a database. After that, familiarity often takes over. If students have had success in a previous use, they are likely to select the same source again, even if “its use in this instance may or may not be appropriate” (according to a small college librarian.) Positive encounters are influential. “If their initial instruction and usage has been successful, I feel they keep returning to what is familiar,” says another (Tenopir, 1999, p. 38). Changes in Citing Patterns. Although there are many opinions among faculty and librarians, only a few researchers have studied what impact these choices have had on scholarship, whether positive or negative. Further research on this topic will help librarians make better collection development choices and will help faculty and librarians design optimal information instruction courses. Philip Davis at Cornell University is one of the few to study changes in the citing behavior of students over time (Davis, 2002a; Davis & Cohen, 2001). Davis and Cohen (2001) found a significant decrease in the frequency of citing scholarly resources in microeconomics undergraduate term papers from 1996 to 1999. Scholarly materials are cited less often and nonscholarly materials such as newspaper articles and Web sites are cited more. The 620 library trends/spring 2003 prevalence of newspaper article citations can almost certainly be attributed to the widespread availability of full text newspaper databases such as ProQuest or LexisNexis Academic Universe through university libraries. The Web may be used from home or on campus. Disturbingly, Davis and Cohen found that approximately only one-half of the URLs cited in student papers led to the correct Web site. An update to this research found that the number of citations to Web sites and newspaper articles in undergraduate term papers continued to increase in 2000, but papers submitted electronically had a higher percentage of correct URLs (Davis, 2002a). Research Methods For Studying These Questions The importance of changes in what research sources are being used and cited is still difficult to judge. Davis (2002a) found no significant relationship “between grade and total number of citations, number of Web citations, number of scholarly citations, or number of non-scholarly citations” (p. 58). Further research is needed to examine the intellectual products created by students and their performance in relationship to the resources used. Studying student preferences and faculty concerns answers only a small part of this important question. Librarians and faculty members can guide students to high quality resources in digital formats as well as in print. More important, but more difficult to understand, is how the inherent properties of a medium influence the learning process (if at all). Most of the concerns by faculty and libraries are related to the quality and appropriateness of the content of digital resources used by students. Quality of content is likely independent of the medium, but undergraduate students may be willing to accept online sources without question. Increased emphasis on teaching evaluation skills, library created portals to authoritative Web sites, and stricter citation guidelines in class assignments are recommended by Davis and Cohen (2001) to help solve this problem in the future. If such suggestions are followed, they should be tested to see if the problem truly lies in the quality of student work or in faculty perceptions of what citations are likely to lead to quality information. Further studies that look at the link between what is cited and the quality of scholarly output will help reveal if a problem truly exists. Academic librarians should work with teaching faculty to conduct such controlled studies. A related research area is to test the effectiveness of user instruction that emphasizes Web site evaluation. Other than course requirements, what truly motivates undergraduate researchers? Is instruction in evaluation methods truly effective or is a more prescribed approach to acceptable resources in assignments necessary? Will student achievement increase after such instruction? Finally, the larger issue of medium and understanding needs more re621 tenopir/electronic publishing search in the scholarly domain, outside of the world of mass media. The Web combines components of mass media with a wealth of scholarly materials making interdisciplinary research appropriate. The work of communications research and information science research dovetails well in this research area. Question Two: New Models for Scholarly Journals— When Libraries Select Electronic Journals, How Do the Products Offered to Them or the Delivery Models They Choose Influence Scholarship and the Way Journals Are Used? Why This Question Is Important Faculty are concerned about what resources students use and cite because there is a long tradition of high quality being associated with peerreviewed, scholarly journals. Scholarly journals provide scholars with a way to convey their own research findings and to keep up with what others are doing in their disciplines. Peer review provides a quality assurance mechanism. The structure of traditional journals has evolved to provide a forum for research, events, and controversies in each discipline. Through such devices as topic issues (bundling related articles on a topic into a single issue), letters-to-the-editor, book reviews, and editorials, journals have evolved to provide more to a discipline that just a body of individual research articles. It can be argued that a journal with a large following in its field, such as Journal of the American Medical Association (JAMA), Reference and User Services Quarterly (RUSQ), or Tetrahedron Letters builds or reflects the community of physicians, or reference librarians, or organic chemists who read it, subscribe to it, and/or publish in it. Traditional scholarly journals continue to be the single most important information source for scientists and social scientists (Tenopir & King, 2000). Today, journals may still be delivered on paper (in fact, a vast majority still use print-on-paper as at least one option), but they may also be delivered digitally, either directly from the publisher or through an aggregator such as LexisNexis, ProQuest, or OCLC. Collection development policies for journals now often favor online versions of journals over print. Digital distribution does not necessarily mean the end of the traditional journal, but it does provide many alternatives to journal article delivery. Publishers may continue to create their journals in issues and merely make each issue accessible on the World Wide Web to subscribers (or for free), individual articles can be separated from the journal issue and be made available in large full text databases that combine articles from many different journals, or authors may choose to post their scholarly work directly on e-print servers or their own Web page and forgo publishing in a journal altogether. Electronic publishing clearly has many advantages for libraries, authors, 622 library trends/spring 2003 and readers, including a speedier publishing/distribution cycle, desktop delivery, and the possibility of wider distribution. The move towards electronic journals is well underway and, while it will move more slowly for some journal titles and some disciplines than others, it is a movement that will continue its momentum (Kling & McKim, 2000). There are several aspects to this research question, answers to which will help publishers, librarians, authors, and readers design the best electronic journals for current and future scholars. We need to know if scholarly journals as we know them contribute more to scholarship and the furtherance of science than separate collections of research articles. If not, databases of separate articles with a good search engine may be the best way for libraries to provide access to the research literature. If so, the scholarly community needs to put pressure on publishers and libraries to continue the journal model. To understand the consequences of such important collection development decisions, librarians and publishers must understand the contributions of a traditional form of a journal (either in print or digital form). A traditional journal that provides scholars in a discipline with a regularly issued bundled collection of related articles plus substantial nonarticle materials such as book reviews, letters-to-the-editor, and an editorial voice includes much more material than a database of articles. We need to understand the extent to which this additional information is valuable to scholarship. Collection development and reference librarians must ask themselves, if we have a good search engine and separate collections of articles is this sufficient for scholarship? Libraries are canceling subscriptions to print and electronic journals in favor of the collection of articles model, often without asking if it matters to researchers and to research. Many times the collection of articles model is offered by a third-party aggregator, while the journal model comes directly from the primary publisher. Choosing aggregate models over journal models may, in the long run, cause smaller journal publishers to cease publication altogether. Perhaps the old model of a journal is obsolete and new models of separate articles distributed through e-print servers, author Web sites, or in databases will replace traditional journals. Librarians now have choices and their choices will help to define the future. These choices should be based on full knowledge of the possible implications of their actions. Previous Work Models for Access to Electronic Journal Articles. To understand the need for research on these questions, it is necessary to understand the current state of electronic journal publishing. Electronic publishing is still in a state of development and experimentation, with many different variations concurrently available. Two main models are competing for subscription dollars and readers’ attention: 1. the journal model, and 2. the article model. 623 tenopir/electronic publishing The journal model retains the structure and purpose of a print journal, but it is delivered in digital form. It is usually available directly from the publisher (either commercial publisher or society publisher) as a Web product. The traditional publishing device of issues and volumes may be retained or articles may be put up on the electronic version as soon as they are ready, without regard to issue numbers. Editorials, letters, and short features likely appear with research articles as part of the electronic journal and a table of contents leads readers to the material. Subscription fees may or may not be charged. All in all, the journal model for electronic publishing retains the sense of community, branding, and editorial voice of the traditional journal, while taking advantage of what electronic distribution and formatting offers. Authors and readers approach an electronic journal much as they do a print journal—by recognition of the journal title as being of interest to them without necessarily knowing what specific information they will find each time they read it. Browsing is the most common method of searching for relevant information and tables of contents with links to the articles are key to browsing. Readers tend to stick to a handful of journals which they consider core to their work. The “SuperJournal” research project in the United Kingdom found that scientists tend to favor the journal model more than social scientists (Pullinger, 1999; Eason et al., 2000; Pullinger and Baldwin, 2002). So-called “journal focused” users look for specific journals, use them to keep up-to-date in their field, and log on to an online journal system every time new issues of their favored journals are loaded. The article model (or “separates” model) is quite different. Aggregators (either publishers such as Elsevier Science or second parties such as BioOne, LexisNexis, OCLC, or ProQuest) take articles from print or electronic journals and make them available in large multititle full text databases. Readers use a search engine to search for and identify articles of interest to them and they read the specific articles out of the context of a complete journal. Pullinger (1999) has called these readers “topic-focused” if they search for articles without limiting to specific journal titles or “article-focused” if they seek to retrieve known articles from a specific journal title. Topic-focused readers likely read articles from a much greater variety of journal titles than do journal-focused or article-focused readers, as found by studies conducted by the OhioLink state consortium (Diedrichs, 2001). A variation on the article model forgoes the first step of publishing in a traditional journal. Authors go directly to an e-print service (such as arXiv.org in Physics or NCESTRL in Computer Science) and post their research papers on this site or post papers on their own or institutional Web site (Pottenger, Callahan, & Padgett, 2001). As with the more traditional article model, readers may rely on a search engine to locate individual articles of interest to their research and the articles are read out of the context of a bundled collection of material that a journal provides. Adoption 624 library trends/spring 2003 of e-print servers varies with workfield, with high-energy physicists notable early adopters (Kling & McKim, 2000). Alternatively, readers may go directly to a known author’s Web site to read separate articles by that author or to a university’s collection of faculty articles in a standard format such as that promulgated by the Open Archives Initiative (Pottenger et al., 2001). The Importance of the Journal Model. Research by Amin and Mabe (2000) reveals why, in an age that offers both print and electronic publishing, authors choose a particular journal in which to publish. The reputation of a journal within its discipline and its reach to the scientific community are important considerations, in addition to factors such as timeliness and responsiveness to authors. These findings echo earlier studies, in a print-only world, by Kochen and Tagliacozzo (1974) and King, McDonald, and Roderer (1981). “Prestige” of the journal is an important characteristic mentioned in all of these studies. Clearly authors have long considered the journal as a branded bundle, where presence within a high-quality journal title lends cache to a research article. ISI’s “Journal Impact Factors” are another important quality factor for authors, editors, and scholars. (See Mabe & Amin [2000] for insights into how Journal Impact Factors are used and misused.) Journal Impact Factors cannot exist without the concept of “journal title,” as an impact score is calculated for each title in a field based on the number of times articles in that journal are cited, divided by the total number of articles published in the journal each year. Many authors seek to publish in journals with high impact factors and scientists in some countries are given bonuses when they publish in these prestigious journals. Much of the research to date has focused on the importance of journal articles to researchers’ work, rather than the importance of an entire journal. Exceptions are the periodic studies of the number of personal subscriptions held by scientists. In his 1974 book, Meadows examined the studies through the early 1970s that discussed personal subscriptions of entire journals, concluding that scientists subscribe to major journals they read regularly either in their research field or general periodicals like Nature or Science. More recent studies show a decline in the number of journals that scientists subscribe to on the average, in direct relationship to the increase in journal prices (King & Tenopir, 2001; Tenopir & King, 2000); however, the purchase of full-text separate articles by individuals did not affect their use of print journals. The number of personal subscriptions has decreased steadily over the last few decades, as the price of scholarly journals has increased. In the past, North American scientists subscribed to an average of six journals each. By the late 1990s this had declined to just over two and the downward trend is continuing. (In contrast to the trend, medical faculty in a recent survey still maintained an average of 6.3 subscriptions per scientist. Two-thirds of all 625 tenopir/electronic publishing of their readings are from their personal subscriptions, both electronic and print; Tenopir & King, 2001.) As the number of subscriptions goes down, scientists rely more on library subscriptions and on separates, from electronic full text databases, paper reprints from colleagues or authors, interlibrary loan, or document delivery services (Tenopir & King, 2000). OhioLink, a large library consortium, offers hundreds of scholarly journals to academic libraries across Ohio. They report that many of the journal titles provided electronically as separate articles were never subscribed to by the libraries that now access them. OhioLink research has found that when such journals are available to faculty and students electronically, the number of journal titles from which they read articles goes up. Beyond those journals considered core to a particular subject, if made readily available, scientists and students read from a variety of sources (Diedrichs, 2001). Recent data from the NorthEast Research Libraries Consortia, however, suggests that the amount of reading from titles that libraries do not subscribe to in print may be less than that found in the OhioLink libraries, with a majority of electronic readings coming from journals that the libraries already subscribe to (Davis, 2002b). What Libraries are Doing. Libraries are spending an increasing amount of their overall materials budgets on serials and an increasing amount of their serials budgets on electronic journals (http://www.arl.org/scomm/ lmbs/lmbs2001.html; http://www.arl.org/scomm/mellon). Some of these are journal model products, such as JSTOR, Project Muse, and SpringerLink. Many are article model databases, such as those from ProQuest, LexisNexis, Ovid, and OCLC. Some combine elements of both, such as ScienceDirect from Elsevier Science. Since budget constraints do not allow much duplication, many libraries are replacing print subscriptions with access to electronic journals. Although libraries have always maintained access to both journal and article models in print collections (Indexes and Abstracts provide access at the article level; journal subscriptions provide access at the journal level), electronic collections are more heavily weighted towards article models. Some libraries also provide links to e-print servers. The PrePrint Network, created by the Department of Energy, Office of Scientific and Technical Information, provides access to many disparate e-print servers (http://www.osti.gov/preprint/). Right now, librarians make the choice whether to purchase a journal model or article model, as both are widely available. If a library changes from a journal model to an article model for electronic journals, will all of the needs of all scholars still be met? Are we, as a recent cover story in Library Journal suggests, in the midst of a revolution in scholarly publishing that means the old models are no longer valid? (Albanese, 2001). Many models will be used and tested—only research (and time) will tell what the effects will be on authors, readers, libraries, and, ultimately, on scholarship. 626 library trends/spring 2003 Research Methods For Studying These Questions Studying the potential effects of changes is a challenge and necessarily must include an element of forecasting. Past research into preferences or journal use can provide some insights into possible methodologies for studying this phenomenon. Two main methodologies have been used to date: 1. surveys to determine preferences of individual scholars, and 2. building prototype electronic journal systems to measure usage and preferences. Tenopir & King (2000, 2001) have surveyed nearly 15,000 scientists and social scientists since the late 1970s to discover a variety of reading variables, including how much they read, characteristics of what they read, what value they get from journal articles, and how much time they spend reading. Questions in the earlier surveys focused, of course, on reading of print-onpaper articles, those from the mid-1990s to the present also include questions to differentiate print and electronic journals. Brown (1999) and others have surveyed faculty within their respective universities to find out how much they read in electronic journals and their preferences for formats. Surveys can create a statistical picture of current habits and, over time, a picture of how things change, which can provide a basis for forecasting future trends. They can reveal differences in preferences among workfields and workplaces. They show how things change (or do not change) over time. They can also be used to measure preferences and readers’ perceptions of and desires for the future, so both qualitative and quantitative data can be collected. Prototype studies focus specifically on electronic journals and can be used to test preferences for specific search or design features. They borrow methods from both focus group studies and system usability studies. Focus group participants can convey preferences and offer ideas for design features. When these features are incorporated into the prototype design, use can be observed in a natural or controlled environment. Pullinger’s (1999) SuperJournal project is the most recent ambitious prototype study; others done in the past include studies by the American Chemical Society and BRS. (See discussion in Tenopir, 1984.) Several measures can be made in prototype studies, including measures of opinions, preferences as measured by use, and other usage patterns within the prototype system. Both qualitative and quantitative data can be collected as users are studied in an experimental setting. Research methods for studying issues surrounding the impacts of various models for electronic journals include surveys, experiments, and observations. These methods yield both quantitative and qualitative data. Qualitative data will give a picture of user preferences, opinions, and forecasts of future behavior. Quantitative data provide information on use, usage patterns, and changes under experimental treatment. Together, qualitative and quantitative studies can provide insights into how journals are now used and will be used in the future. These insights will help us under627 tenopir/electronic publishing stand the implications of collection development decisions by librarians and publishing decisions of both publishers and authors on scholarship. Question 3: Intermediaries—Are Librarians— As Intermediaries to the Search Process— Still Necessary in a Digital Age? Why This Question Is Important The articles model of electronic journals relies on effective search strategies entered into reliable online systems. Since the early days of online searching in the 1970s, researchers have studied search strategies, first of expert searchers and later of novices, and observed the interactions of search intermediaries and their clients. These studies have aimed to develop optimal search strategies and procedures that could be incorporated into online systems to make them easier to use and build on the many decades of studying how reference librarians interact with library patrons. It may seem odd to place a phenomenon that has been studied for so long on a list of research topics for the future, but researchers have reached the point with this topic that they are able to build on a firm foundation and compare future research with an accepted body of research from the past. This is not always the case with library and information science research topics. The ultimate goal of many information retrieval system designers is to create a system that will be so user friendly and “intelligent” that it can be used independently, so users can answer their information needs with a minimal amount of assistance. (Drenth et al., 1991). The intermediary may not often conduct an entire online search for a user anymore, but even the traditional helping roles of the reference librarian are being questioned in a digital environment. Many users search online without seeking help from a reference librarian and spend many hours on the Web without ever considering asking for assistance. Even if human assistance is no longer being sought by many users, librarians need to understand what, if anything, a human intermediary or reference librarian contributes to the information search process, which of these contributions can be captured in system design, and which must remain an individualized human process. We need to understand which of these contributions can lead not only to better systems and better library services, but also perhaps to justification for funding librarian services. Previous Work The literature review here is not comprehensive, due to the vast quantities of articles on this and related topics. Several review chapters in the Annual Review of Information Science and Technology (ARIST) (see, for example, Bates, 1981; Borgman, 1984; Harter & Hert, 1997; Drenth et al., 1991) and an early literature review by Fenichel (1980–81) describes in depth 628 library trends/spring 2003 related research literature on topics of search strategies, online system evaluation, and expert system design. Lynch (1978) studied the reference interview process in public libraries in a mostly preonline era, and Bunge reviewed the traditional reference interview literature in 1984. The current discussion includes only some of the studies that specifically address the contributions of the intermediary in the online searching process. Seminal Work. Even a limited discussion of this topic must include mention of several seminal studies that form the basis for much of the work that came after them. Most modern studies of the intermediary-client interaction point to Taylor, who, in 1968, differentiated between when users seek information on their own and when they approach a librarian. Asking for assistance usually only occurs late in the search process, and reference librarians are skilled at using questions as filters to elicit the information needed to understand an information need. Bourne (described in Hawkins, 1982) was one of the first to observe strategies of expert online searchers, placing strategies into several broad categories. Citation Pearl Growing, Building Blocks, Successive Fractions, etc., are now recognized general approaches to searching that have been tested and extended by Bates and others (Hawkins, 1982). In 1979 Bates developed a set of search tactics that are used to record and measure specific steps used in online searches of standard Booleanlogic based systems. Soon after, Fidel (1985) empirically developed a set of operational and conceptual moves used by intermediary online searchers. These specific moves and tactics are still used today to measure search strategies by both novices and information professionals. Having two accepted measurement scales makes research easier to conduct and to compare, so much of the research on search strategies conducted from the 1980s on uses the work of Bates or Fidel. Borgman (1989) examined in depth how individual differences and human characteristics influence choice of search strategies. Borgman found that the time to reach success in computer-related tasks, such as online searching, has a much larger variance than in other types of tasks. Technologically inclined people are able to complete search tasks much more quickly than others and have fewer problems with search techniques such as Boolean logic. (An entire branch of information science research has evolved around the human factors study of the cognitive side of information seekers [see Ingwersen, 1999; Pettigrew, Fidel, & Bruce, 2001]; it is much too broad to review here.) Questioning. The questions asked by both users and intermediaries, and how they are asked, have been shown by many researchers over time to be a crucial factor in the search process and search success. Some early important studies of the human aspects of interaction with information systems show why questions are so important. Belkin (1980) observed that users approach a system due to a gap in their knowledge. 629 tenopir/electronic publishing Belkin expressed this as ASK—anomalous states of knowledge—and demonstrated that it is difficult for users to express an information need or ask a question when they have such gaps in their knowledge or understanding. Human intermediaries may, through the reference interview, help users get to the point where the ASK can be resolved, but online systems do not yet do this very well (Belkin, 1980; Belkin, Oddy, & Brooks, 1982). Information seeking online involves a range of human capabilities, including the cognitive, affective, and sensori-motor domains (Nahl & Tenopir, 1996; Wang, Hawke, & Tenopir, 2002). All three domains interact and influence each other. Most systems and documentation confront only the cognitive and sensori-motor domain questions (e.g., how do I conduct a search and what button do I push?). Human interaction can better address the affective domain (e.g., will I make a mistake?) than help screens or documentation do. The role of each domain has been identified in studying questions asked during the search process. Nahl and Tenopir (1996) and Tenopir, Nahl-Jakobovits, and Howard (1991) found that novices ask many questions when they are searching for information online, and, when no intermediary is present, most of these questions are never verbally expressed. Using the technique of protocol analysis, the researchers tape-recorded novice faculty, undergraduates, and graduate students while they searched for information in a general interest full-text periodicals database. Participants were instructed to “think aloud” as they proceeded, voicing their internal thoughts. Analysis of the transcribed recordings revealed that, on the average, novices had many questions per search, an average of eleven moves per search (as measured by Fidel’s categories). If a reference librarian had been nearby (or available online), at least some of these questions would likely have been asked and answered. Questions may get verbalized when a human intermediary is present. Wu (1992) recorded the many questions asked of intermediaries by search clients both in the presearch interview and, more frequently, during the search. Although no one session contained all categories of questions, Wu observed nine categories of questions (elicitations), including questions related to: search terms, search procedures, databases, output, and other information services. Both Wu (1992) and Nahl & Tenopir (1996) observed a high percentage of reassurance questions (called “echoic” by Wu and “rogering” by Nahl & Tenopir)—questions that needed no answer (for example, “are you sure?”), but are basically part of the affective side of human interaction. The interaction between an intermediary and a client can take on many dimensions. Spink, Goodrum, and Robins (1998) studied interactions both in the presearch interview and during the search session. They found that intermediaries requested fifteen different types of information from their users, including requests for information about search strate630 library trends/spring 2003 gy and terms, output (relevance), domain knowledge, and database selection. (Wu, 1992, and Nahl & Tenopir, 1996, examined the other side of questioning in this process—the questions posed by end users during the search process.) Spink et al. (1998) studied interactive feedback between intermediaries and academic clients during presearch interviews and online searching sessions. Data collected and analyzed included videotapes, transcripts, online searching logs, and relevance judgments by the users. This gives a good picture of the various types of interaction that occur between an intermediary and a client during a search and shows how rich that interaction can be. Spink discovered a total of 885 interactive feedback occurrences in 40 mediated online searches, with a mean of 22 per search. She further categorized these occurrences into five types: content relevance feedback, term relevance feedback, magnitude feedback (size of output from a query), tactical review feedback (display the search strategy history), and term review feedback. White has studied the reference interview over time (1985, 1989, 1998). In her 1998 study she found that the information specialist dominates the presearch interview, but both clients and intermediaries tend to ask shortanswer questions that are mostly focused on the subject, search strategy, the service, and the output. The most common types of questions were verification questions (approximately half of all questions). Verification questions are similar to the echoing or rogering questions so commonly held by users, in that the questioner already knows a supposed answer, as opposed to questions that elicit totally new knowledge. Several researchers have studied how different types of questions posed by reference librarians influence the satisfaction and perceptions of users. (See, for example, Auster & Lawton, 1984; Allen, 1988; Dervin & Dewdney, 1986; Dewdney & Mitchell, 1997; and Radford, 1996.) Findings show that, although both open-ended and closed questions are asked during the interview process, users are more responsive to open-ended questions and questions that probe into why something is needed, rather than close ended or simply factual questions. Spink et al. (1998) observed “complex interactions” between users and intermediaries throughout a search session. Whether the questions are posed by an intermediary to a user or from a user to the intermediary (or are merely in the minds of the users), the online research process is one in which numerous spoken and unspoken elicitations occur. Robins (1998) found that the mediated online search process is highly interactive and that intermediaries and users work together in a nonlinear interplay both during the presearch interview and during the search. The search topic even changes focus on the average approximately every seven utterances. Systems today do not yet recognize this important interaction. More research is needed on the effect question asking and question answering have on the success of online research and on the satisfaction levels of users. 631 tenopir/electronic publishing Most of the studies mentioned above tape-recorded interactions, with the researchers working from transcripts of these recordings. Transaction log analysis is another method for gaining insights into the searching behavior of novices. Many researchers in the past decade have analyzed the transaction logs from searches of library catalogs, commercial online systems, or Web search engines. (See for example, Hunter, 1991; Peters, 1989; Zink, 1991; Saracevic & Kantor, 1988; and Spink et al., 2001.) Wallace (1993), for example, examined transaction logs of the online catalog from public access terminals at the University of Colorado libraries. She found variability in the amount of use the search aids available on the system. Quick search and express search features were used rarely; search history and searching other databases were used more extensively. Catalog users prefer to search for natural language keywords and then scan lists of titles. Information Overload. These studies show that intermediaries help answer factual questions, provide reassurance to affective questions, and help users clarify their information needs. Another role that librarians as intermediaries play is to help users select the best resources and overcome information overload. Information seekers today are much more likely to find too much on their topic rather than too little because most of our online systems are better at locating lots of information than locating only the best information. Extensive studies of Web search behavior by Spink, Wolfram, & Jansen (2001) found that average users of a Web search engine enter only a little over one term per search and review only the first page or two pages of the results. This either means that the relevance ranking algorithm in the Web search engine works so well that the users find everything they need on the first screen or, more likely, they are overwhelmed by the thought of going beyond twenty or so items and decline to delve more deeply. Many librarians have called for reference librarians to help end users sort through the vast quantities of information to locate the best sources for their needs (Hopkins, 1995; Rice, 1989; and Biggs, 1989 for example). Hopkins suggests several possible ways this might be accomplished, including quality filtering by ranking by citation counts or citations in review articles, structured guides to the literature that include quality judgments, and better information literacy instruction. Teaching Search Strategies. User instruction is another traditional role of reference librarians. Many librarians have commented on the increased need for instruction with the prevalence of independent end user online searching (see for example, Tenopir & Ennis, 1998). Mercado (1999) surveyed how instruction programs have changed and recommended that library users need to be taught not only how to search, but critical thinking skills. Beyond the specific techniques of systems, librarians are finding that students need to know overall search approaches and how to choose which databases to search. 632 library trends/spring 2003 Developing search strategies is not a natural skill. Taylor and Penhale (1998), for example, found that, even after instruction, students at Earlham College still needed librarians to help them devise and refine complicated searches. Students may have unrealistic expectations of their searching ability and what information they can find on their own. Librarians, through instruction and real-time assistance can help inject some reality as well as skills (Tenopir & Ennis, 1998). Searching is more of an art than a science, however. Saracevic and Kantor (1988) studied search strategies used by experienced searchers on the same topics. They found much variation in the terms selected for searching and the search formulations. Critical thinking skills, rather than the one best way to search, will help make more independent searchers, but intermediary assistance can help users think about alternatives. Research Methods For Studying These Questions Information professionals play many roles in the search process, not only that of an intermediary conducting a search for a client. The roles of question-answerer, affective reassurer, instructor, presenter of alternatives, and quality filter have all been found to be a part of search success. Most of the research into these roles and the effect on search success can fit under the general heading of “user centered” research. The methods and methodologies of user-based research are well defined and described by Wang (2001). Observations, protocol analysis, and transaction logs together provide a baseline of mostly qualitative data. In the future, additional observational studies will build on past research, but observations should be combined with experimental techniques. Observational studies generally rely on qualitative data and are often described as “preliminary” or “exploratory.” This is appropriate when a topic is new, but now that specific factors and behaviors have been observed and measures developed, traditional experimental techniques can explicitly test these factors. In the future, we need to discover in more depth how search behaviors vary with different intermediary behaviors and with systematically introduced variations in help screens, real-time online help by librarians, and documentation. Similarly, user satisfaction and the effect on the intellectual output of students and scholars should be measured against the introduction of different treatments in the mediation process. With such a rich baseline of exploratory studies, it is now time to produce more definitiveexperimental work. ReferencesAlbanese, A. (2001). Revolution or evolution: Trends in electronic scholarly publishing. Li-brary Journal, 126(18), 48–51.Allen, B. L. (1988). Text structures and the user-intermediary interaction. RQ, 27, 535–541.Amin, M., & Mabe, M. (2000). Impact factors: Use and abuse. Perspectives in Publishing, 1. 633tenopir/electronic publishing Auster, E., & Lawton, S. B. (1984). Search interview techniques and information gain as ante-cedents of user satisfaction with online retrieval. Journal of the American Society for Informa-tion Science, 35, 90–103.Bates, M. J. (1979) Information search tactics. Journal of the American Society for Information Sci-ence, 30, 205–214.Bates, M. J. (1981). Search techniques. In M. E. Williams (Ed.), Annual Review of InformationScience and Technology (Vol. 16, pp. 139–169). White Plains, NY: Industry Publications, Inc.Belkin, N. J. (1980). Anomalous states of knowledge as a basis for information retrieval. Ca-nadian Journal of Information Science, 5, 133–143.Belkin, N. J., Oddy, R. N., & Brooks, H. M. (1982). ASK for information retrieval: Part I. Back-ground and theory. Journal of Documentation, 38(2), 61–71.Biggs, M. (1989). Information overload and information seekers: What we know about them.The Reference Librarian, 25/26, 411–429.Borgman, C. (1984). Psychological research in human-computer interaction. In M. E. Williams(Ed.), Annual Review of Information Science and Technology (Vol. 19, pp. 33–64). White Plains,NY: Industry Publications, Inc.Borgman, C. (1989). All users of information retrieval systems are not created equal: An ex-ploration into individual differences. Information Processing & Management, 25, 237–239.Brown, C. M. (1999). Information seeking behavior of scientists in the electronic informationage: Astronomers, chemists, mathematicians, and physicists. Journal of the American Soci-ety for Information Science, 50(10), 929–943.Bunge, C. A. (1984). Interpersonal dimensions of the reference interview: A historical reviewof the literature. Drexel Library Quarterly, 20, 4–23.Davis, P. (2002a). The effect of the Web on undergraduate citation behavior—A 2000 update.College & Research Libraries, 63(1), 53–60.Davis, P. (2002b). Patterns in electronic journal usage: Challenging the composition of geo-graphic consortia. College & Research Libraries, 63(6), 484–497.Davis, P., & Cohen, S. (2001). The effect of the Web on undergraduate citation behavior: 1996–1999. Journal of the American Society for Information Science & Technology, 52(4), 309–314.Dervin, B., & Dewdney, P. (1986). Neutral questioning: A new approach to the reference in-terview. RQ, 25, 506–512.D’Esposito, J. E., & Gardner, R. M. (1999). University students’ perceptions of the Internet:An exploratory study. Journal of Academic Librarianship, 25(6), 456–461.Dewdney, P., & Mitchell, G. (1997). Asking “why” questions in the reference interview: A the-oretical justification. Library Quarterly, 67(1), 50–71.Diedrichs, C. P. (2001). E-journals: The OhioLINK experience. Library Collections, Acquisitions,and Technical Services, 25(2), 191–210.Drenth, H., Morris, A., & Tseng, G. (1991). Expert systems as information intermediaries. InM. E. Williams (Ed.), Annual Review of Information Science and Technology (Vol. 26, pp. 113–154). Medford, NJ: Information Today, Inc.Duggan, A., Hess, B., Morgan, D., Kim, S., & Wilson, K. (1999, April). Measuring students’ atti-tude toward educational use of the Internet. Paper presented at the Annual Conference ofthe American Educational Research Association, Montreal, Canada. ED429117.Eason, K. D., Yu, L., & Harker, S. (2000). The use and usefulness of functions in electronicjournals: The experience of the SuperJournal project. Program, 34(1), 1–28.Fenichel, C. H. (1980–1981). The process of searching online bibliographic databases: A re-view of research. Library Research, 2, 107–127.Fidel, R. (1985). Moves in online searching. Online Review, 9, 61–74.Grimes, D. J., & Boening, C. H. (2001). Worries with the Web: A look at student use of Webresources. College & Research Libraries, 62(1), 11–23.Harter, S. P., & Hert, C. A. (1997). Evaluation of information retrieval systems: Approaches,issues, and methods. In M. E. Williams (Ed.), Annual Review of Information Science andTechnology (Vol. 32, pp.3–94). Medford, NJ: Information Today, Inc.Hawkins, D. (1982). Online bibliographic search strategy development. Online, 6(3), 12–19.Herring, S. D. (2001). Faculty acceptance of the World Wide Web for student research. Col-lege & Research Libraries, 62(3), 251–258.Hopkins, R. L. (1995). Countering information overload: The role of the librarian. In J. B.Whitlach (Ed.), Library Users and References Services: Special Reports (pp. 305–333). New York:Haworth Press. 634 library trends/spring 2003 Hunter, R. N. (1991). Successes and failures of patrons searching the online catalog at a largeacademic library: A transaction log analysis. RQ, 30, 395–402.Ingwersen, P. (1999). Cognitive information retrieval. In M. E. Williams (Ed.), Annual Reviewof Information Science and Technology (Vol. 34, pp.3–52). Medford, NJ: Information Today,Inc.King, D. W., McDonald, D. D., & Roderer, N. K. (1981). Scientific journals in the United States:Their production, use, and economics. Stroudsburg, PA: Hutchinson Ross Publishing Com-pany.King, D. W., & Tenopir, C. (2001). Using and reading scholarly literature. In M. E. Williams(Ed.), Annual Review of Information Science and Technology (Vol. 34, pp. 423–477). Medford,NJ: Information Today, Inc.Kling, R., & McKim, G. (2000). Not just a matter of time: Field differences in the shaping ofelectronic media in supporting scientific communication. Journal of the American Societyfor Information Science, 51(14) 1306–1320.Ko, H. (2000, August). Internet uses and gratifications: Understanding motivations for using the In-ternet. Paper presented at the 83rd Annual Meeting of the Association for Education inJournalism and Mass Communication, Phoenix, AZ. ERIC Document ReproductionService No. ED444208.Kochen, M., & Tagliacozzo, R. (1974). Matching authors and readers of scientific papers. In-formation Storage and Retrieval, 10, 197–210.Lin, C. A. (1999). Uses and gratifications. In G. Stone, M. Singletary, & V.P. Richmond (Eds.),Clarifying communication theories: A hands-on approach (pp. 199–208). Ames: Iowa StateUniversity Press.Lubans, J. (1998). How first-year university students use and regard Internet resources. Re-trieved March 19, 2003, from http://www.lib.duke.edu/lubans/docs/1styear/firstyear.html.Lynch, M. J. (1978). Reference interviews in public libraries. Library Quarterly, 48, 131–138.McBride, K. B., & Dickstein, R. (1998). The Web demands critical reading by students. TheChronicle of Higher Education, 54(28), B6.McLuhan, M. (1964). Understanding media: The extensions of man. New York: McGraw-Hill.Meadows, A. J. (1974). Communication in science. London: Butterworths.Mercado, H. (1999). Library instruction and online database searching. Reference Services Re-view, 27(3), 259–265.Nahl, D., & Tenopir, C. (1996). Affective and cognitive searching behavior of novice end-us-ers of a full-text database. Journal of the American Society for Information Science, 47(4), 276–286.Peters, T. A. (1989). When smart people fail: An analysis of the transaction log of an onlinepublic access catalog. Journal of Academic Librarianship, 15(5), 267–273.Pettigrew, K. E., Fidel, R., & Bruce, H. (2001). Conceptual frameworks in information behav-ior. In M. E. Williams (Ed.), Annual Review of Information Science and Technology (Vol. 35,pp. 79–113). Medford, NJ: Information Today, Inc.Pottenger, W. M., Callahan, M. R., & Padgett, M. A. (2001). Distributed information manage-ment. In M. E. Williams (Ed.), Annual Review of Information Science and Technology (Vol.35, pp. 79–113). Medford, NJ: Information Today, Inc.Pullinger, D. J. (1999). Academics and the new information environment: The impact of lo-cal factors on use of electronic journals. Journal of Information Science, 25(2), 164–172.Pullinger, D., & Baldwin, C. (2002). Electronic journals and user behaviour. Cambridge, UK: DeedotPress.Radford, M. L. (1996). Communication theory applied to the reference encounter: An anal-ysis of critical incidents. Library Quarterly, 66(2), 123–137.Rice, J. (1989). The hidden role of librarians. Library Journal, 114(1), 57–59.Robins, D. (1998). Dynamics and dimensions of user information problems as foci of inter-action in information retrieval. Proceedings of the 61st ASIS Annual Meeting, October 25–29,1998, Pittsburgh (Vol. 35, pp. 327–341).Saracevic, T., & Kantor, P. (1988). A study of information seeking and retrieving, II: Users,questions, and effectiveness. Journal of the American Society for Information Science, 39, 177–216.Spink, A., Goodrum, A., & Robins, D. (1998). Elicitation behavior during mediated informa-tion retrieval. Information Processing & Management, 34(2/3), 257–273. 635tenopir/electronic publishing Spink, A., Wolfram, D., & Jansen, B. J. (2001). Searching the Web: The public and their que-ries. Journal of the American Society for Information Science and Technology, 52(3), 226–234.Taylor, N., & Penhale, S. (1988). End-user searching and bibliographic instruction. In T. B.Mensching & K. J. Stanger (Eds.), Bibliographic instruction and computer database searching:Papers presented at the Fourteenth Library Instruction Conference held at Eastern Michigan Uni-versity May 8–9, 1986 (pp. 57–74). Ann Arbor, MI: Pierian Press.Taylor, R. (1968). Question-negotiation and information seeking in libraries. College and Re-search Libraries, 29(3), 178–194.Tenopir, C. (1984). Full text databases. In M. E. Williams (Ed.), Annual Review of InformationScience and Technology (Vol. 19 pp. 215–246.) White Plains, NY: Knowledge Industry Pub-lications, Inc.Tenopir, C. (1999). Database use in academic libraries. Library Journal, 124, 36–38.Tenopir, C., & Ennis, L. (1998). The impact of digital reference on librarians and library us-ers. Online, 22(6), 84–88.Tenopir, C., & Ennis, L. (2001). Reference services in the new millennium. Online, 25(4), 40–45.Tenopir, C., & Ennis, L. (2002). A decade of digital reference: 1991–2001. Reference & UserServices Quarterly, 41(3), 264–273.Tenopir, C., & King, D. W. (2000). Towards electronic journals: Realities for scientists, librarians, andpublishers. Washington, D.C.: Special Libraries Association.Tenopir, C., & King, D. W. (2001). The use and value of scientific journals: Past, present, andfuture. Serials, 14, 113–120.Tenopir, C., Nahl-Jakobovits, D., & Howard, D. L. (1991). Strategies and assessments online:Novices’ experience. Library & Information Science Research, 13, 237–266.Tenopir, C., & Read, E. (2000). Patterns of database use in academic libraries. College & Re-search Libraries, 61(3), 234–246.Wallace, P. M. (1993). How do patrons search the online catalog when no one’s looking? Trans-action log analysis and implications for bibliographic instruction and system design. RQ,33(2), 239–252.Wang, P. (2001). Methodologies and methods for user behavioral research. In M. E. Williams(Ed.), Annual Review of Information Science and Technology (Vol. 34, pp. 53–99). Medford,NJ: Information Today, Inc.Wang, P., Hawk, W. B., & Tenopir, C. (2002). Users’ interaction with World Wide Web resources:An exploratory study using a holistic approach. Information Processing and Management,36(2), 229–251.Wei He, P., & Jacobson, T. E. (1996). What are they doing with the Internet? A study of userinformation seeking behaviors. Internet Reference Services Quarterly, 1(1), 31–51.White, M. D. (1998). Questions in reference interviews. Journal of Documentation, 54(4), 443–465.Wildemuth, B. M., de Bliek, R., He, S., & Friedman, C. (1992). Search moves made by noviceend users (1). Proceedings of the ASIS Annual Meeting, 1992 (pp. 154–161).Wu, M. M. (1992). Information interaction dialog: A study of patron elicitation in the infor-mation retrieval interaction. Unpublished doctoral dissertation. School of Communica-tion, Information and Library Studies, Rutgers University.Zink, S. D. (1991). Monitoring user search success through transaction log analysis: TheWolfPAC example. Reference Services Review, 19(1), 49–56.